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In these days where "accountability" is the byword, or¬ 
ganizations more and more frequently seek to evaluate 
their programs. They often hire outside evaluators to help 
them assess the effectiveness of their programs, to find 
out what works and what doesn't, and to determine 
what programmatic changes would be beneficial. 

Acknowledge, a New York City action research or¬ 
ganization, is one such evaluator. In this paper, we 
examine an evaluation we conducted of an after¬ 
school program operating in New York City public 
schools to reflect on the use of focus groups as a 
means of evaluating afterschool programs. Since the 
administrative office that hired us to do the evalua¬ 
tion and the directors of the program we evaluated 
wanted to learn more about the young people 
enrolled in the program and the staff that runs it, the 
use of focus groups as a research method seemed a 
logical choice. The literature on focus groups as a 
methodology provides a rationale for using this tech- 
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nique although it also suggests some of the challenges. 
To this research, we bring our own experience in evalu¬ 
ating the afterschool program, discuss the challenges we 
encountered in using focus groups, and conclude with 
suggestions for future work involving focus groups in 
afterschool evaluation. While we reflect here from the 
point of view of evaluators, we hope that this article will 
be useful to program staff and administrators, as well as 
fellow program evaluators, so that full and ethical part¬ 
nerships between the numerous stakeholders involved 
in evaluation work can be fulfilled. 

Use of Focus Groups in Research and 
Evaluation 

Focus groups have been a popular research method in 
the social sciences since the 1980s (Asbury, 1995; Bader 
& Rossi, 2002; Krueger &r Casey, 2000; Morgan, 1996; 
1997; Smith, 1995). Focus groups typically consist of a 
small group of six to twelve partic¬ 
ipants who have some salient char¬ 
acteristic in common, such as 
belonging to a particular program. 

One or two trained facilitators mod¬ 
erate the discussion and encourage 
participants to share their opinions 
and experiences (Asbury, 1995; 

Krueger & Casey, 2000; Morgan, 

1996, 1997). Besides asking ques¬ 
tions, facilitators are also responsi¬ 
ble for bringing the discussion back 
on topic if it loses focus. As Morgan 
(1996) notes, focus groups are different from everyday 
group conversations in that the purpose is to discuss a 
particular phenomenon, reaction, or experience. The 
emphasis is on the interaction the group creates (Mor¬ 
gan, 1996). This emphasis on group interaction is what 
differentiates focus groups from individual interviews. 
The discussions that emerge during focus groups allow 
researchers to explore a topic in greater depth than is 
possible with some other instruments such as surveys. 
Facilitators’ ability to ask participants to clarify certain 
areas of discussion allows the facilitators to better inter¬ 
pret focus group findings (Nabors, Reynolds, & Weist, 
2000 ). 

Wilkinson (1999) notes that the interactive nature 
of focus groups addresses a number of problems of 
social research, including the possibility that the 
research can ignore and thereby reproduce power imbal¬ 
ances, that it may be looking at phenomena out of con¬ 
text, and that it produces artificial accounts of people’s 


lived experiences. By tapping pre-existing groups such 
as a group of staff members, focus groups can gather 
information specific to the workplace. In focus groups, 
the information shared is produced in the same social 
context that the evaluation is trying to understand. As 
group members talk out their agreements and disagree¬ 
ments, researchers can observe and document both what 
information is shared and how that information is 
socially constructed. 

Recently, focus groups have become a popular 
method in program evaluation (Christie & Rose, 2003; 
Morgan, 1996, 1997; Smith, 1995). Fitting focus groups 
into existing program structures, such as staff meetings 
and youth councils, has advantages, since it brings the 
research into the social and physical setting it aims to 
understand. It also enhances the potential focus groups 
have for creating a collective consciousness within a 
group about the political forces and resource structures 
in which group members operate. 
This consciousness, apart from for¬ 
mally articulated research findings, 
can be a critical motivation for cre¬ 
ating positive change. 

Because focus groups are easily 
adaptable to different settings and 
cultures (Morgan, 1996, 1997; see 
Balch & Mertens, 1999) and 
because this method is particularly 
beneficial for participants with dif¬ 
ferent perspectives from those of 
adult facilitators (Bender & 
Ewbank, 1994), program evaluators are increasingly 
conducting focus groups with young people who are 
participants in the program being evaluated. Evaluators 
and researchers in general praise the use of this method 
with youth because it actively involves them in the 
research process and values their feedback (Kitzinger, 
1995; Morgan, Gibbs, Maxwell, & Britten, 2002). This 
insight led the evaluators of the Core Arts program in 
Mississippi to conduct separate focus groups with pro¬ 
gram staff and child participants to explore the pro¬ 
gram’s successes and areas of difficulty (Harvard Family 
Research Project, n.d.). In New York City, Thompson 
(2005) conducted focus groups with children enrolled 
in an afterschool fashion program to learn about their 
experiences. 

Despite the rosy picture the social science and eval¬ 
uation literature paints of this technique, implementing 
focus groups in program evaluations can also have draw¬ 
backs. For example, “groupthink,” the phenomenon in 
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which participants conform to the consensus of a group 
rather than voicing their individual opinions and con¬ 
cerns, can occur during these discussions (MacDougall, 
1997). Fitting focus groups into existing program struc¬ 
tures can also have disadvantages. As we will illustrate 
below, difficulties with sampling and participant selec¬ 
tion can alter the outcome of the discussion and, in turn, 
color evaluation findings. Further, 
conducting focus groups in existing 
groups of staff can reproduce power 
imbalances. Preexisting tensions or 
internal alliances, invisible to 
researchers, can limit the honesty 
and depth of discussion. 

Context 

Fhe afterschool program we evalu¬ 
ated was operated by a community- 
based organization (CBO) in public schools in 
low-income New York City neighborhoods. We reflect 
here on the second year of a three-year longitudinal eval¬ 
uation in which we collected program information 
through surveys and observations in addition to focus 
groups. The afterschool program typically served one- 
quarter of each school’s population and offered acade¬ 
mic support, academic enrichment, and youth 
development programs, such as sex education and 
decision-making curricula, as well as fun activities such 
as theater and dance. In addition to these activities, the 
afterschool program also provided students and families 
with such resources as health and social services, deliv¬ 
ered either by the CBO or by other organizations with 
which the CBO had developed links. 

Students in the program faced multiple barriers to 
school success: The majority tested below state and city 
standards, almost half spoke Spanish at home, all were 
exposed to violence in their communities, and many faced 
other family issues. Academic support and enrichment 
were thus key activities in the afterschool program, which 
sought to make learning fun and engaging for students. 

Challenges of Using Focus Groups in 
Afterschool Evaluation 

Our experience with this evaluation both illustrates the 
challenges of using focus groups with young program 
participants and suggests ways to address those chal¬ 
lenges. We used focus groups as an evaluation tool 
because the sponsoring organization and its program 
directors wanted feedback from program participants 
and staff. Focus groups, because of the advantages of the 


technique outlined above, were chosen as a means to 
gather such feedback through meaningful dialogue 
among participants and staff. 

The focus groups were conducted in the middle of 
the school year so that students and staff who were new 
to the program had sufficient time and experience to 
build opinions about it. In focus groups with young peo¬ 
ple, we explored their experiences 
by asking which aspects of the pro¬ 
gram they particularly liked or 
wished to change. Focus groups 
with program staff discussed issues 
related to youth development as 
well as the challenges and supports 
they encountered in their work. 

Although we have used focus 
groups successfully and extensively 
in other projects, we were surprised 
to discover that conducting focus groups with adults 
and young people in afterschool settings was not as easy 
or straightforward as we had anticipated. Though we 
continued to find that focus groups were a valuable eval¬ 
uation tool, we also faced challenges with a number of 
issues related to logistics, ethics, and sampling. 

Physical Setting 

We found that the physical setting in which a focus 
group is conducted can strongly influence its progres¬ 
sion and outcomes. In this evaluation, we conducted 
focus groups with students and staff in school cafeterias, 
libraries, teacher lunchrooms, offices, classrooms, 
kitchens, and staff lounges. Each setting carried its own 
built-in behavior program, which we had to consider 
and sometimes modify. For instance, when we con¬ 
ducted focus groups in classrooms, youth would often 
raise their hands instead of just jumping into a conver¬ 
sation. Social norms about who usually uses a space and 
what activities are allowed or forbidden are communi¬ 
cated both by what people already know about the space 
and by such physical attributes as furniture. 

Although moderators can and do temporarily alter 
focus group settings, any space has physical limitations, 
some of which are easier to manipulate than others. The 
open echoing space and long, narrow, benched tables of 
a cafeteria, which limit the interaction and privacy of a 
group discussion, are difficult to change. Whenever pos¬ 
sible, we used a space such as a teachers’ lunchroom, 
where students are usually not allowed. Such a space not 
only helped ensure privacy but also indirectly let young 
people know that we valued their comments and took 
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their suggestions seriously, because we conducted the 
group in a room usually reserved for adult staff. 

Privacy 

Hand in hand with concerns about space were concerns 
about ensuring privacy. Once a focus group begins, 
interruptions disrupt both the content of conversation 
and the sense of cohesion among participants. However, 
when doing this kind of research with young people, the 
need to create and maintain privacy is complicated by 
practical concerns about who is responsible for the 
young people’s safety and how they may react to an 
unfamiliar adult. During this evaluation, we asked pro¬ 
gram staff to leave the room when 
we were talking with young people 
in order to allow them to speak 
more candidly. In a few cases, par¬ 
ticipants tested our facilitation skills 
by disengaging from the conversa¬ 
tion, being disrespectful to other 
youth, or walking out of the room. 

These incidents highlighted the 
need to balance privacy with prac¬ 
ticality and safety. 

Privacy is also extremely 
important for focus groups with staff, because their com¬ 
ments could affect their employment or their subse¬ 
quent interactions with—and trust of—supervisors and 
peers. During this evaluation, more than one program 
director ignored our request for privacy—including, in 
one case, a sign posted outside the room—and entered 
the session with program staff mid-discussion. In these 
cases, staff sometimes expressed that they were upset by 
the lack of respect they experienced. Similarly, we felt 
frustrated because the interruptions disturbed the flow 
of the focus groups and indicated that the program 
directors were not taking our efforts seriously. Such dis¬ 
ruptions thus provided valuable information about the 
context of a program, which we captured in the docu¬ 
mentation and considered in our analysis of the discus¬ 
sion. From experiences like these, we learned to speak 
with program staff, and especially with supervisors, 
beforehand about the arrangements for the focus groups 
and to agree on ground rules to ensure privacy. Similarly, 
at the outset of every focus group, we discussed with 
participants what they hoped to learn, what we would 
and would not do with the material shared, and what 
limiting factors or concerns group members felt. 


Sampling 

Though much has been written on ethics in social sci¬ 
ence research, Smith (1995) notes that relatively little 
has been written about the ethics of focus groups, 
despite their increasing popularity. Social scientists gen¬ 
erally adhere to specific ethical responsibilities including 
respect for autonomy, which means that research partici¬ 
pation must be voluntary; non-maleficence, or the 
researchers’ obligation not to inflict harm; and benefi¬ 
cence, or consideration of the benefits, risks, and costs of 
participation (see Beauchamp & Childress, 1994). 

Because we take these principles quite seriously, we 
could not ignore the ethics of our work as we talked to 
youth and staff about their after¬ 
school programs. We often con¬ 
fronted practical constraints, group 
dynamics, developmental consider¬ 
ations, and institutional power 
dynamics whose ethics we were 
forced to navigate more or less on 
our own, since the literature does 
not provide guidance on these 
issues (see Smith, 1995). 

Though the selection of partic¬ 
ipants for focus groups would most 
commonly be considered a sampling issue, it raised eth¬ 
ical concerns as well. In our search for a group of young 
people who would be willing to talk to us, we naturally 
turned toward the program directors. We hoped they 
would pick groups of engaging youth who were not 
afraid to share their views. While this method of choos¬ 
ing participants was practical, we wondered after the 
fact how we could assure these youth that their identity 
was protected when they had been hand-selected by the 
very directors whose programs they were critiquing. 
Aside from concern about possible sampling bias that 
could affect the validity of findings, we pondered how 
we could honestly tell youth that they should share their 
concerns openly. In retrospect, we realized we should 
have been more honest with youth about what protec¬ 
tions we could or could not provide so that they could 
decide which opinions they wanted to share about their 
programs. Respect for autonomy includes giving all indi¬ 
viduals, youth or adult, the information necessary to 
make informed decisions. 

Participant Confidentiality 

Another ethical concern was confidentiality. Participants 
in a focus group can reveal information about other 
group participants. As facilitators, we always mentioned 
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that anything shared in the group should not be shared 
outside the group. However, we also had to clarify that 
we would have to tell a staff member if any participants 
expressed harmful thoughts such as wanting to injure 
themselves or others. A transparent introduction to the 
focus group protocol can accurately reflect the extent to 
which the information shared in the focus group is avail¬ 
able to people not located in the immediate setting. In 
our case, we explained that we would write reports sum¬ 
marizing what everyone said in the focus groups with¬ 
out identifying any individual participants by name. We 
also reminded youth that, although we would be shar¬ 
ing the group’s input with the program, no one in 
the group should share outside the group any¬ 
thing a particular participant said. 

Staff Support 

Other experiences in this evaluation 
emphasized the importance of having the 
support of the program staff. Staff sup¬ 
port can consist simply of helping set up 
a room or providing extra paper and pens. 

More importantly, staff support is vital to 
the care and safety of the young 
participants. In one instance, we 
arranged for a staff member to be 
nearby while we conducted a focus 
group with youth; however, when 
we needed help finding a partici¬ 
pant who suddenly walked out, the 
staff person was nowhere to be 
found. Addressing this situation 
took up a lot of the time allotted for 
the focus group and disrupted not only the flow of dis¬ 
cussion but also the group’s sense of safety. While we 
thought we had taken precautions beforehand, we 
learned that we should have explored support logistics 
and expectations with staff much more clearly. 

Such experiences taught us that evaluators and pro¬ 
gram administrators need to be clear with one another 
about the logistical requirements for conducting suc¬ 
cessful focus groups. Evaluators need not only to share 
what a focus group is, how it operates, and how it con¬ 
tributes to the overall evaluation, but also to engage all 
staff members who assist in scheduling and organizing 
the groups in discussion of the requirements for running 
a smooth focus group. Similarly, evaluators must accom¬ 
modate the lived realities of the organizations they eval¬ 
uate. For example, we sometimes had to accept that a 
director chose a group of young people for our focus 


groups based on the fact that those youth were free dur¬ 
ing our meeting time, though we wanted to include dif¬ 
ferent groups of youth as well. Evaluators and program 
staff need to find a balance between methodological 
idealism and realistic practicality by communicating in 
advance about arrangements for the focus groups. We 
learned to ask staff and administrators what they would 
like to learn from the focus group. Reflecting on possi¬ 
ble positive outcomes for the program provided an 
incentive for staff to ensure that the group went 
smoothly. 

Youth Development and Safety 

Following the principles of positive youth 
development increasingly adopted in after¬ 
school settings meant shifting our ethic 
about the goals of the research and raised 
additional concerns about ensuring 
youths’ safety. 

We learned to view focus groups not 
simply as a way to extract data, but as 
group activities that could promote posi¬ 
tive youth development ideals. In the after¬ 
school setting, the principle of 
beneficence (Beauchamp & Chil¬ 
dress, 1994) needed to include pos¬ 
itive youth development ideals such 
as encouraging the young people to 
participate actively on multiple lev¬ 
els and helping them to feel valued, 
safe, and supported. Integral to 
positive youth development is a 
respect for the importance of youth 
opinions and the significance of their knowledge in cre¬ 
ating quality youth programs. We found that open com¬ 
munication with the youth helped them feel valued from 
the beginning. We used transparent introductions to 
make it clear that both we and the program administra¬ 
tors wanted to hear the young people’s thoughts and 
opinions in order to make better decisions about pro¬ 
gramming. Laying ground rules about how to respect 
others’ opinions also helped to ensure a safe space. Curs¬ 
ing at others or interrupting peers was discouraged from 
the beginning and reinforced consistently throughout 
the group meeting. We engaged the young people in set¬ 
ting the ground rules and in other aspects of facilitation 
in order to gain their investment in the process and out¬ 
comes of the groups. For example, besides participating 
in the ground-rules discussion, participants also were 
engaged in note-taking and in such aspects of group 
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moderation as ensuring that our conversation had one 
speaker at a time, stayed on topic, and kept within our 
time limit. 

In our experience, problems with youth disrespect¬ 
ing each other or the facilitator mainly arose when a 
child had been forced to participate. We therefore told 
program staff that we wanted young people to partici¬ 
pate in the focus group of their own accord. Young peo¬ 
ple’s participation is helpful and important only when it 
is voluntary; a focus group will not yield useful results 
if it produces feelings of coercion and frustration. 

What if an argument or a physical fight does break 
out in the focus group setting? When we ran into such 
precarious situations, we had to “decide if and how to 
intervene” (Smith, 1995, p. 483) in each instance. 
Although we had to be prepared to act, we also coordi¬ 
nated with a staff person to be available if the need arose. 
To ensure privacy, we tried to have a staff person located 
outside the room, or available by cell phone or walkie- 
talkie, in case a child wanted to leave or we needed help 
in controlling the group. Addressing this logistical issue 
prior to conducting focus groups helped alleviate stress 
not only for the youth, but also for us and for program 
administrators, who were concerned about the youths’ 
safety and the program’s liability. The need to have a staff 
member available reinforced the need to establish a clear 
understanding with staff about what we might need 
while facilitating focus groups and what the program 
could provide. We learned that it was more effective to 
reschedule a group when we became uncomfortable 
with the logistical arrangements than to continue in 
order to finish all groups on a timetable. 

Some of our discussions with youth and staff were 
dominated by two or three individuals, while other par¬ 
ticipants felt uncomfortable speaking in the focus 
groups or were not able to express themselves verbally 
due to language barriers. Multiple avenues for partici¬ 
pation, another youth development concept, helped 
address this problem. In our focus groups, we tapped 
into multiple intelligences through drawing, mapping, 
writing, and role play in addition to guided collective 
conversation (Morgan, Gibbs, Maxwell, & Britten, 
2002). Both adults and youth can participate in such 
activities, which serve to mediate power differentials and 
create a comfortable atmosphere (Yuen, 2004). When 
working with immigrant youth, we addressed language 
issues by ensuring that the facilitator was bilingual. 
These methods allowed all participants to express their 
opinions so that we could capture vital information that 
might otherwise have been lost. 


Following our philosophy that youth and staff 
should continually inform the evaluation process, we 
provided opportunities for participants to offer feedback 
to facilitators at the close of focus group meetings. The 
Participatory Action Team in New York (Zeller-Berkman, 
in press), a group of youth researchers, used this tech¬ 
nique in conducting focus groups with other youth; 
their findings have led to important insights about and 
improvements in protocols and facilitation techniques. 
Simply extracting data without attention to process is 
not in line with the ethics of the positive youth devel¬ 
opment that guides our work and that of the programs 
we serve. 

Flexibility of Methods 

Even with good communication and preparation, we 
found we had to be ready to adapt protocols and be flex¬ 
ible in facilitation strategies when plans changed. For 
example, one staff focus group conducted as part of a 
regularly scheduled staff meeting had an unusual high 
attendance of 25 people. Conversely, a similar focus 
group at another site involved only five participants. 
Though the ideal focus group is six to twelve people, 
evaluators can facilitate a productive group of a different 
size if they are prepared to change the focus group pro¬ 
tocol. When we met with the large group, we shifted 
from our planned strategy of talking in one group to 
using a cluster of breakout groups along with writing 
exercises. At intervals during the focus group, smaller 
groups shared major themes of their discussions with 
the other groups. This combination allowed staff mem¬ 
bers to talk to one another about their ideas and expe¬ 
riences and still captured individual thoughts on paper. 
Focus groups with fewer than six people are challenging 
because participants tend to speak to the moderator 
rather than to one another. In our group of five staff 
members, we emphasized that participants should use 
questions or probes and came up with techniques to 
encourage them to do so. For example, the moderator 
can sit down among the participants rather than stand, 
so that the group focuses less on the perceived role of 
the moderator. Another strategy is to flip statements 
directed to the moderator back to the group by asking, 
for example, “Do you all agree?” With these techniques, 
we found that the few participants engaged in a mean¬ 
ingful discussion rather than simply providing short 
answers for the researcher. 

We sometimes used role play and dramatizations as 
facilitation strategies in our focus groups with youth. 
Flowever, we found that these techniques worked best 


Schaefer-McDaniel, Libman, Zeller-Berkman & Krenichyn 


FOCUSING IN 31 


at sites where the afterschool programming included 
drama or theater activities. When young people had 
experience using performance as a communication tool, 
the role plays were engaging, fun, and informative. At 
sites where youth were less accustomed to this kind of 
activity, the young people sometimes found the use of 
role play frustrating, confusing, and even draining, so 
we had to find other ways to engage them. 

Incentives 

Since we were asking young people to take time away 
from their programs to talk with us, we wanted to com¬ 
pensate them for their time. While such compensation 
may not be common practice in evaluation work, social 
science research with children and young people gener¬ 
ally recommends providing young participants with 
incentives (see Morrow & Richards, 1996). Interestingly, 
the literature on focus groups does not discuss the 
nuanced consequences of this practice; this apparently 
simple decision brought on new 
discussions in our team. We needed 
gifts for a rather large group of 
young people but operated under a 
tight budget. We were thus forced 
to ponder whether we should pro¬ 
vide each participant with a present 
or pool the money to buy a DVD or 
a pizza party for each program. We 
also worried about how young peo¬ 
ple who were excluded from the 
group would feel about not receiv¬ 
ing gifts. After much back and 
forth, we decided to give individual 
gifts only to participating youth, 
which meant that we were not able 
to spend much money on each 
present. We wondered how the 
young people would interpret our 
simple present of a school supply 
item. Would they be excited that we 
gave them a gift or offended because 
it was not very expensive? While most of the youth 
seemed content with their presents, some commented on 
the fact that the pen they picked was on the cheap side. 
Our experience with incentives on a low budget was thus 
mixed and inconclusive; our own solution was to go back 
to our funder to ask for enough money to purchase bet¬ 
ter gifts in subsequent phases of the evaluation. 

Another challenge with the incentives was the ques¬ 
tion of when to distribute them. We gave them at the 


start of the focus group, using the activity of grabbing a 
gift out of a bag as an icebreaker. We thought that giv¬ 
ing presents early on would show youth that they were 
not required to participate in order to “earn” their gifts 
and let them know that we valued their participation. 
Unfortunately, we found that giving presents at the 
beginning all too often distracted youth as they com¬ 
plained about their gifts or attempted to trade with other 
participants. 

Program Change 

After listening to the concerns youth and staff were voic¬ 
ing, we often asked ourselves, “What is being done with 
the recommendations?” While organizations initiate 
evaluations in order to improve their programs, they 
vary in their ability and willingness to implement the 
changes that an evaluation report suggests. We were ask¬ 
ing youth to share their opinions in order to inform their 
own programming with no assurance that their input 
would in fact be used. To address 
this concern, we went above and 
beyond standard evaluation proto¬ 
cols to ensure that the programs 
responded to the findings. For 
example, we often created brief 
intermediate reports to be distrib¬ 
uted earlier than our traditional 
report at the end of the year or 
term. We sometimes helped staff 
sort through recommendations to 
see which were feasible and to cre¬ 
ate an action plan for implementing 
changes. The evaluation feedback 
became a dialogue between the 
evaluators and the program staff. 

Lessons Learned 

We value the use of focus groups 
with afterschool program staff and 
participants because such groups 
allow dialogue, provide information 
that we cannot explore through close-ended surveys, 
and give youth, in particular, the chance to express their 
experiences. Because use of focus groups with young 
people is a fairly new methodology, we found ourselves 
learning good strategies for engaging young people “on 
the fly.” Perhaps the most important strategy we discov¬ 
ered was the need to be flexible. 

We learned several valuable lessons through our 
qualitative evaluation work with afterschool program 
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staff and youth. First and foremost, we discovered that 
conducting focus groups is not as straightforward in 
afterschool settings as in some other contexts; it requires 
a deeper level of preparation than simply arranging for 
time and materials. We needed a definite plan of action, 
considering such questions as: Flow many participants 
do we need? Where can we conduct the focus group 
with minimal interruption? What is our goal for this 
focus group? Evaluators and program staff should work 
closely together to discuss needs and concerns, particu¬ 
larly those related to space, safety, and privacy. Focus 
groups are more likely to be successful when thought 
goes into creating a space that is trusting, open, and safe. 

We learned to explain our agenda and set rules for 
discussion at the beginning of each focus group with 
youth. To do so, of course, we had to ponder these rules 
with program staff prior to the session, carefully exam¬ 
ining which rules were important and why. Young peo¬ 
ple also can and should weigh in on behavioral 
guidelines for focus groups, which may be different 
from those appropriate in other spaces such as class¬ 
rooms and program activities. In a related point, evalu¬ 
ators and program staff should have a clear plan of 
action in case something goes wrong, for example, if 
youth start fighting. 

As an agency, ActKnowledge is reflecting on ways 
youth can participate in evaluation beyond simply being 
part of a focus group or other evaluation technique 
(Krenichyn, Schaefer-McDaniel, Clark, & Zeller- 
Berkman, in press). On ffart’s (1992) continuum of 
youth participation, we are still in the beginning phases 
of creating opportunity for deep youth involvement. We 
are pushing ourselves to find ways of including youth as 
more active, responsible participants in the research 
process, for instance, by involving them in research 
design. We hope fellow evaluators and other programs 
will join us in the attempt to involve youth stakehold¬ 
ers not only as participants but also as co-researchers in 
the evaluation process. 
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